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Theatre auditions by 2M semifinaljLSts in the 1980 
Scholara in the Arts program were evaluated under two conditions, 
four judges ranked the live auditions^ irhile five evaluated 
videotapes of the sase parforaance of the high school seniors. The 
auditions were videotaped in black and white. A single cameia was 
used, fixed at an in teroiediate distance so as to give a wtiDie-body 
view* Tapes were loailed to the judges for viewing on playback 
equipfflent* Ccrrelations between the two sets of assessments indicated 
consideraole agreeeaent across methods of evaluation, though they 
were not eguivalent to one another. In the present sample^ the use of 
videotapes for final decisions about nominations for awards would 
have resulted in a completely different group of nominees rrom those 
actually chosen. On the other hand^ videotapes used as a screeniag 
device would have permittad a 60S reductinn in those invited to give 
liva auditions with a loss of only one individuai who was eventually 
determined to be among or near the five most accomplished* Despite 
the limitations of this effort, its results point clearly to the 
usefulness of videotapes as a screening device, (Author/RLI 
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Theatre auditions by 24 high school Heniors were evaluated under 
two conditions. Four Judges ranked the live auditions, while five 
evaluated videotapes of the same performance. Correlations between the 
two sets of assessments Indicated considerable agreement across methods 
of evaluation, though they were not equivalent to one another. Videocapes 
could he useful as a screening device, serving to limit the number of 
Lndividuals to be seen in live audition in an adjudication program. 
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Kvalu.iCi.ni or Video tapt?d and Livt? Ihoarre Audit ion ?s 



in ^oholarw in the Art a prouranu almost ^Ot.) hiuh sohool 

- -- iM'!^- 'A'vilVK'. Wi^r.- ^-/i''v,f.'J j two- 

-: .i:.:t.^ pr n-t-H-.. Aro 1 : ,\iL ions and lorters oi reo.Tmrvndat ion vera us^id to 
-t.A%'w L /-* Ht:^::u 1 i na I i s c ^ ; tiit/ St*m 1 f ina I i s Cs^ ueve then ^een in auditioiu 
\[-t- woro evontiiallv r^o^^-iinendt'd to tiic I'ros Idt^nt ial Sonolars OkMj-.ml io-n 
: ' ^- i vc- awa rd?^ - 

■hi^. r roooa uro ^Iven ririe to two concerns ==ono of quality and one 
c^^r^c, lii^i iiid>:os showed ^Kcellent agreement with one another in their 
, .1 luot ions or the auditions; thus, there is little reason to t[ueHtlon 
vh- vilidity their choices froni anoni; those who reached semiflnarist 
staru=;. However, there Is no way to know whether excellent candidates 
wore rnLssed in the first stage ot selection, owing to an inability to 
orv^^-nt their qualifications effectively on papar. And, in fact, within 
^ht- rUTU f ina 1 is c qroup there was no relation between the final ranking 
o" the atuiltiiMis and the preliminarv ranking of candidates based on 
iPo 1 ioat tons ; the written inaterials, therefore, do not appear to provide 
i wr: ^M'od has is ror predicting actual performance, In addition, live 
n;dUi^i'ns iro expensive. If the program grows a^ expected in future 
veirs, travel costs associated with bringing any subst..atial percencage 
^\ Liiw .ippIit.auLs ti'gethur for audit L.ais will overwhelm anticipated 
Lnano 1 n^ * 

Bo til these concerns might be met in part by the use of videotaped 
auditinns as part of the application procedure. Isniatever their flaws, 
video* tapes c(^uld provide information .about performing ability that cannot 

it 



be obtained fron wrlccen natarial^. "f evaluations of tapes ri!iouId 
provti sufticlantly siniilar to evaluations based on live auditions, ic 
would be possible to liniiL se^^vrely tht* nunibur .if i HlI i v i duvi 1 ^ nr/itt'd 
to audicion livo while scill havinc hphio an^^ur inco tha: -nit ^ t and i ni.i 
candidates werR not excluded iron tha final aHso^Rnont* 

Beforo initiat LniZ cho iiso ^^f capori, howL'\'or, wo Hiuuild .dualn 
oonf irmac ion chat chov do provido a v'alid basis for oval nation. I'iiis 
repi'Tt describes a pre 1 imina r%' exam ina 1 1 oii of tho rolatUMi botwota^ 
assessments based on live auditions and chose based lui vldtri'iapos ot chit- 
same no r f orniance . 

Mecjiod 

The 2^ Scholars in the Arts auditions constitute the performance 
aval ua ted. Each student was ^iven four minutes in which to [ire sent two 
selections, unrestricted except that no more than one selection could 
be an original work. Props were limited and elaborate costuming was 
discouraged. The auditions were videotaped in black and white, 
A sinfle camera was used, fixed at an interTriediate distance so as to 
give a ^hole-body view* 

Five adjudicators agreed to view the videotapes; tlieir namOH and 
affiliations are given in Attachment A. They were told that the purpose 
of the evaluation was to determine whether ''...we can make effective use 
of secondary media as a screening device... to select a very small number 
of applicants-- perhaps 2-3% of the total group---fpr the live auditions 
from which the award winners will be selected." They were given a roster 
llscing the performers along with the names of their selections, and a 
second roster on which students were to be ranked ■'...from 1, most outstanding 

f 
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in .uvomp 1 Lr^hnit^nt and promise, tc 24, lease ouc^cand inu No furchLT 
definition of che HoLeotion oritt?rLa wan offered, and no di root i.-nn woro 
uivon a- to a.^u thi- i ipt?si ririould he viewed. Tapes wore naiiod lo tiu- 

'ud^es for viewinc on choir own or rented plavback oqulpment. 

Kesiil 

:hro- jadoos oonip! led fullv with tiio ranking ins t ruo c i ^mih . The t\'c 
a i ii Ln.: prof w-rrod to j:roup taiLi oandldater^, laie Ur^inu four oatoi;orios, 

ind :ho •■thor, five. I iod ranks wero arisigned in those latter two oaseH* 
aad rrt' : at 1 .'ns ar:onA I I the seLs of rankings were v'^bcainod. These 
/or re' lEU^ns, sho\sTi in che upper portion of Table 1, ran^o fror: 

. ". to .3^^ witih a median value of .3b. The reliability of nn .sessment 
ohrainod averatiini; aoross five ludges is .73 (coeffioient alpha), 
'f hur^ , whLIt' tht^te is only moderate: a£;reement anunig independent assessments 
:oiSc-d on \-ideotapes, a score derived by averaging* over the entire set of 

;au.^^-s is saffioiontly reliable to merit consideration, 

Fhe evaluations of the live auditions are showTa in the middle 
portion of rable I. Correlations for the four live judges are based on 
tiioir in Jo pen dent rankings of the oandi dates prior to discussion and 

iss i v.uiniont of final audition ranks, The oorrelatiDns range frnm ,46 to 
.SO, with a median of .63; the reliabilitv of a score obtained by averaging 
over judges is .88. Thus the judges who viewed the live Darformance agreed 
with one annther soniewhat more strongly than did those who worked from 
V idoi;' tapes . There Is, howcvar, no basis for concluding that evaluations 
of live performance are inherently more reliable than those of videotapes-'-- 
rfiere is n<^ wav, with the present dtita, to diHtingulsh effects due to 
videotnping from possible differences among die Judge.s^ or to take account 



Ji^curfs Che canaiJjt.'H whilu reviowin.: HuMr app 1 1 rat : on -att/riaU. 

Kelationri ht^tWi-ai Uit» :vo >^^'C^ t^%'a 1 uat i nnn aia' nhown in iiiu 
lover pcrcton of tao Lahu- :he jorr.4aLion^ raiU'o fron -.1^ lo ,{-)^, 
an.! hav-;^ a rio^M aa .^a '''a^., t-^> avoraro, tno o-aluation^ of 

videocapoj p^^^^i'or-^an jo a-roo wita -a-'^o a !^ livo po r ^"o r-Mac 'o arirl'^' 
rhfe? a.irio .lo.iroe aa rho%' i/iaa^ ar:or;i' r at^:":^o ] \a^r^ , 

Two roral ao^i^roa v'crt ,:-Minoa, .-au :ha- \a.auHape .^valuations anJ 
on^^ tor liv^ t'va] aat i oa:^ , l^v -^iianiao .>vor ail r^'lovant ]u,iantMitH= liiOHO 
two Hot-, of -o^rc- .■orrv!a:i/ ,^T^---:.ioh oia^uja t^^ fndioato oonsidorablo 
agr^^omont ^icro^n r:L'taoJ ^--f o\'a I aat i on , taouk^li ntU so h i cli as to Indioato 
that those r^othods are ofnn%'alorir to ono anotiior. The ostlmatod triio 
sooro oorrolacf on fn^tweon rho two sots of rankings Is .70. 

Thwre is ^o-.: sua^ostlo:: in the oorrolatioa^ that two of tho assosF- 
mantH based on vidootnpos diffor fron the remainine throe in clieir 
relatLuns to the live iLid^r;ents. rankings by jiid^es VI and V4 correlate 
suhstan t Lai Iv with the j iid Lni^enr s of live auditiors (median r's of .51 and 
,37), whllo rankitu^s hv tiio renuiininu three videoLape Judi^es have lower 
relations tc^ thr^se iud^ments (rodlans of . 3^ , ,2^^, and .20). Factor analy- 
sis confirns thU difference. f-rinoiDal axes factor analvsis suegasts two 
correlatGd factors in the iudsments (r^ ^ ,1RK All the judgments of live 
auditions alonR with thoso bv 1 udge VI define one factor, while those by 
lud^eB V2 , VI and V5 define a second factor. Evaluations bv judge V4 have 
their principal loading on the second factor along with a secondary loading 
on the i'ir*^t. 



Anocher wav look at ch^ relations bt?cwr^t?n cht} two sots oi valuos 
is exannnf J 1 f r L»ru' 08 in ranklnus rcooiveJ bv indivuiii.il oaiui i do t. o-^ 
fr.^in I'Ht^ kirui '.-f e^^aluatipn to Che? othor, ( riu- ooniploto not rankiiu^.s. 
Ln/lodint; thorfo bv indK'ldual jud^os a^ wo I 1 ,0^ lIio two total HOt^roH, 
^ 00/^' a La A L L joai:u,Mit B,l rirst:, opus^idor too iivo individuals wh.^ 
Aara- uo l ua 1 1 \' ni^nunauod t.^ neoonif award wLtuwrH, MstL^d in laait^t lO 
: ■ \ r initial Iivo audition rank;^ (1, Z, 5, and n), thoso individuals 
tMok^^i In too vlao-c ipL* aHHOr<H";t'nr . ^, 1''^ an.! rivis , 

vidu^'tui'ad norfortnanoo had boon usdd to mako a final so loot ion of 
a.^r^inijoa, ni-no or thosie who wera actual]^' nominated would iiavo beon 
i^'vii tr'o avsarLl winnor?^. On tho pthor hand, if vidootai^o tad botui 

\ 'Vt^i t.^ aoU'Ot :5omi f inai isCH for audition, it would havo boon neooH = 
'^Av 10 Hid i: ion I i\'0 onlv 10 of the 2^ to view all but ono of thoe^o who 
t^vc^ntualLv leoolvod nomlnationH (and, in fact, these 10 would have 
in ^ laded all but one of the cop eight as judged from live auditions), 
liiat IS, a ^0' reduction in the number of individuals auditionod would 
a.i^ r' Ta'-aittd in substant in I 1 v the same Mroup as those who reoeivt d 
a i na t i ons .^r wore oloso to doing so, but with the loss of one individual 
\=;- w :s t-/ontuallv placod in the top five. 

It is r.^towc^rth^ Ehat there was one very large discrepancy in the 
iw..^ suus of rankint;s involving an individual who was near the bnttom of 
t:u.* ranks ir the live judi^lng (a rank of 21) but who wa& placed first in 
tp.e vl don tape judging. Four of the five videotape assessors agraad that 
this indLvidual was the most accomplished of those seen (or, for judges 
whi^ E?rvMiped candidates, was one of those in the highest-rated group), 
Jnoludlny this case, there were five instances of substantial discrepancies 
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in Che two total rankings (dLffarences of eight or more in^s i t; ivMir^ . ) 
Two involved individuals evaluated as more aoonmplishod in the video 
than in the live judgmencri, while three involved the reverse, 

A numbar of possible sources of difference between tho twc^ nets or 
avaluaclons can be hypothec ized , One possibilitv is a fortuitous one: 
Videotape judges may have been forced to discount differences in diction 
and projection because of limitations imposed bv the relativelv poi^r 
sound quality of the tapes. The writer's preferred speculaclon is that 
those who were given superior rankings in v^ldeotape than in live 
iuogings were generally those who displayed a high level of energy and 
motion and whose movements shoved poise and fluidltv. Perhaps tliese 
two possibilities are related, in that the lack of good informaLlon 
concerning some aspects of performance would force greater reliance 
on those aspects which are clearly conveyed on the tapes. There is 
insufficient evidence available to test these hypotheses. Coniments 
by the videotape judges suggested only that those whose videotaped 
Derformance received better evaluations than their live performance 
were percaived as particularly fresh, imaginative, innovative; while 
those whose live auditions received better evaluations were seen as more 
conventional in characterization or in choice of selection* 

Several of the judges, in addition to comments on individual 
students, offered reactions to the conditions under which performance 
was videotaped or to the use of videotape In general. Their comments 
are sunmarized in Attachment C. 
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Discuss io n 

iiii-^ ^tudv h.M ?^hov^; thdt t?va 1 iiac itMis baHtHl on vidLMUaped audit, an 
are relaLed, hut not idt^nLical, Co evaluations ot tl^e live auditlimH 
:>o::: whijh the tarerf are or gained. In the present sample, the use of 
^ ' - 1 ^ ^ ^ ^ a : . . ..i i ol! i .^^ i i ui ^ aL-t'u r iiom Ina L i on^ i o r aworuH wou i l! iia ve 

'i • '.iij i^-^v^/ di-li^i^ji^L ^r>.''ii[' i?i noiiziaooEs lyoir^ Luv^se i:it'tu.,ili'/ 
-- -t-n. On :;ie othor hand, \'ideota[^es u^^ed as a screeniiii; device would 
'. o.>' o^^r^Tiicted a ntr reduction in tlio^^o invited to tJtve live audition'^ with 
i 1-^---^ vO -^alv ono indLx^idual who was e%^entuaLly deterniined to be anions; 
= -r a^-o'' live inont a^^ oonn I Lshed . 

!n 'O^nHidering the magnitude of the reLacion cibrierved "between the 
:W' r^z'i^ of valueH. r^everal facton^i miclpating apainHt pood agreement 
-^a^'UL^.: ije btirne in mind. Firnc, the comparison of methods of 'Valuation 
Wis CO, rrlod cMjt on a select group. Involving between six and seven 
i'tTcent of che program applicants. If the use of applications and 
r-jc-VTimendat ions to ch(iose semi f inal is ts had any validity at all, this 
criMi;} was more homA\aeneous in its range of talent than was the pool of 
i:' -Iic.ints from which it was drawn; we would expect stronger relations 
^M-rwtu/n the two Hets lU tis ssmen t s If they were applied to the larger, 
mc-re JiverHo population. 

Soc^aid, th^^ quality of the vidOiTtapes emplt^yed may have affected 
t:io results. The vLdeotaping had been carried out under restrictive 
c editions so as not to interfere with the live judging of the auditions; 
;n r^art Icultir , only ome microph.one, fixed some distance from the stage, 
wa::? ^lermitted. The sound track on the tapes was poor, and several judges 
thought the lighting could have been improved* It is likely that better 
tapes would be produced if they were seen as an Important part of the 
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evaluation rather than as an adjunct to it. Better tapes could weir= 
make the task of evaluation ^mote' reliable , and might affect the basis 
on which evaluations are made*-for example, by allowing greater weight 
to be given to a student's vocal qualities. 

Finally, the asBessment of videotapes was carried out by judges 
who had no opportunity to work together to achieve agreement on their 
bases and standards for evaluation; In contrast, those who judged the 
live performance had spent many hours working together to select a group 
of semif inallsts* It is likely that a similar opportunity to influence 
one another's thinking would result in greater agreement among those 
evaluating videotaped performances and this in turn would permit the 
Emergence of a stronger relation to judgments of the live auditions. 

This study was a preliminary effort. It would be desirable to 
repeat it with a number of refinements^ For example: (1) An entire 
applicant pool^ rather than those preselected on some basis, should be . 
sampled; this would allow an assessment of the relations between live 
and videotape evaluation across a broader range. of talent* (2) The same 
judges should be involved in both methoHs of assessment. A design could^ 
be developed In which each Judge viewed half of a group live and half on 
videotape; It would then be possible to determine relations between 
methods while "holding constant" any differences among judges* (3) Judges , 
should be asked to provide systematic evaluations of a number of aspects , 
of a candidate's performance as well as an overall evaluation. We coul^ "/; 
then seek to discover whether there are particular dimensions of accomplish- 
mmt which are differentially weighted depending on the medium in which , 
evaluation Is conducted* j 



Despite the limitations of this effort. Its results point clearly 
to the usefulness of videotapes as a screening device. Experience with 
candidate-produced videotapes and more systematic study along the lines 
suggested above will be needed* however, before we can be certai^ as to 
just how severe a selection is desirable at the screening stage and 
just how closely videotape and live assessments can be brought into 
agreement with one another. 
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Table 1 

Correlations Among Rankings by Individual Judges 



Correlations Among Rankings of Videotaped Auditions 





V2 


V3 


V4 


V5 


VI 


.38 


.21 


.42 


.18 


V2 




.59 


.34 


.32 


V3 






.38 


.23 


V4 








.51 



Correlations Among Rankings of Live Auditions 
L2 L3 L4 

LI * 77 .60 ,80 

L2 .46 .67 , 

L3 .60 

Correlations Between Videotaped and Live Audition Rankings 





VI 


V2 


V3 


V4 


V5 


Ll 


.65 


.35 


.18 


.S3 


.18 


L2 


.67 


.30 


.28'' 


.60 


.29 


L3 


.40 


■ .13 


.23 


.22 


-.17 


L4 


.40 


.30 


.33 


.64 


.22 



# Note I With 24 cases a correlation of ,51 is significant at 
the 1% level of confidence^ while one of ;.41 is significant at the 
5% level. 



If 



Vld«dtap« Adjudieatdrs 



Yale Scheel of Drua/ 
Yale Repertory Theatre 



Profeaaor Arthyr Storehp Chalrara 
DepartMnt of Draaa 
Syracuee University 



Frefeasor Mirlaa Tulla 
DepartMiit of Drma 
Hofstra Unlveralty 



Hinry A, Wickep Jr.t OiairMii 
DepartMSt of Theatre 
Director » Division of the Arts 
nie Packer Collegiate Institute 



George Wojtasikg Hanaglng Director 
Equity Library Theatre 
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Attachment B 

Rankings by Individual Judges and Pooled Over Judges 

Live Auditions . Videotaped Auditions 

Pooled LI L2 L3 L4 Pooled VI V2 V3 V4 

1 1 4 5 1 9 7 11 10 9.5 

2 3234 6 48 19 4 
.3 5 1 6 3 8 2 16 6 9.5 

4 4 8 1 5 10 8 * 5 11 14,5 

5 2946 2 10 12 34 

6 8 5 2 7 16 14 18 8 9.5 

7 10 3 15 9 3 9 l' 5 4 

8 9 ^ 10.. 10 10 4 5 4 2 14.5 

9 6 13 ^ 11 11 12 13 6 21, 9.5 
10 11-6 17 12 15 11 19 22 4 
11^13 20 ^ 12 ^ .2 5 16 ^ 7 7 4 
12^ 15 12 16 8 17.5. 18 23 12 4 
13. 7 11 21 13 11 1 9 15 14.5 

14 16 18 9 14 24 15 21 23 21 * 

15 20 7 14 20 . 7 6 -10 9 l4.V 

16 18 17 8 L9 / 21 23 - 17 14 21 
17.5 12 15 ^3 16 19 19' 14 20 21 
17. 'S - 19 19 13 15 17,5 22 3 / 4 21 ■ 

19 17 21 7 23 13 3 13 13 14.5 

20 14 14 22 22 14 'l2 15 18 14.5 

21 21 16 24 17 1 17 2 1 4 

22 24 23 18 18 23 20 24 24 21 

23 ' 23 22 20 21 20 24 22 17 21 

24 22 24 19 24 22 21 20 16 21 



I, 



-15^ 
Attachment C 
CommentB by Judges (Selected and Edited) 



' Dn the Uba of Video tapes 

Vi: The good performers still, jumped out, and the poor performers also 
were obvious. The middle students could have been a tossup for 
many reasons-^f at Igue , the video, the lengthy yiewing, etc, liy 
sense is that video might prove to be a liable option to live 
audition if good results are obtained In this evaluation, 

V2 ; I did not like^using the tape. I felt distanced from the event 

and from the candidates. This did not ^help ^me to be objective, It 

kept me from relying on subjective responses I have leapned'**to 

trust and use* The TV medium Is too cool; it removes the synergistic 

putential of live theatre*. Only In the case of the most dynamic 

^>erformer was his actual presence felt on the screen. That would 

be fine If one were Judging professionals, but not for novic^^^s 

who have low development ofskills* 4 

On These Tapes 

VL: The audio range , was limited when an actor moved out of cenjter . stage 
There was an Interffcrlng hum when the operator raised, the volume 
Co cry to record low voices. The sta^e flDor whs a drum,, hurt ing 
one student who hurled liimself about^ ^ . ^ ^ % ^ " 

The harsh white light prevented clear - facia'l definition, washing 
out faces* There was little or no Jight on the apron of the stage, 
BO chat faces were in darkness when an actor was kneeling. 



V2 1 The qualitv of the sound was quite poor* For example, one student's 
' physical work was quite .free and he used most of the stage, but the 
stage noise created by the movement made the words inaudible at 
times. 

The camera moved only across the stage but never at anything closer 
Chan a medium shot. Therefore, the features of the candidates were 
not sharp, I actually could not determine what an actor was experl^ 
encihg, except through an aural evaluation, 

V3: If the quality of the audio portion of the tape could be improved, 
it would facilitate the adjudication* ^ 

V4 : This series of auditions has many flaws. It may seem that a single 
long-shot setting approximates the view of the judges as they sit 
facing the stage. Actually, I found the "technical-tedium" of this , 
frame and the poor sound a very real handicap. It is possible to 
do a much more acceptable job by learning how and when to use a zoom 
lenS"So that even one camera can give some variety* The TV coverage 
must be much better^-indeed , can be so within a few hours of training. 
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Attachment- C continued 

V5 : I believe the process could be improved if after we see a full 
length shot of the auditioner introducing himself the camera 
could move in much tighter to a "bust" s_hDt--^so that the viewer 
would be able to see the expressiveness of the auditioner's face. 

The quality of the lighting is poor and the sound could be 
improved . 

Other Consents t Method of Rating 

V3-. I first viewed the tapes straight through to get an Idea of the 
level of talent and for a first subjective opinion. Then I 
developed a series of criteria upon which to make a judgment. 
These crtteriaj all given equal Importance, were: A- Selection 
of Choice (appropriateness and variety); Stage Presence 
(posture, movement, execution); C- Projection (diction, clarity); 

Control (delivery, energy, pace, attack). Each of the two 
selections of each candidate was graded Individually and then 
totaled up for a preliminary score. In case of very close scores 
a third viewing was done and the selection process was made by my 
. own subjective impression* 

Other Cominents: Choice of Material 

V4: The choice of audition material was, on the whole, quite good. 

However, adaptation of scenes for solo presentation was sometimes 
very clumsy and it would be advisable to inform teachers and 
coaches to give better editing advice in this area. Physical 
appearance was kept suitably simple^-I felt that only one student 
was actually hindered by costuming. 
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